Welcome to the

NBIS Neural Nets & Deep Learning workshop

Who are we?

Claudio Christophe Bengt Per
Claudio Christophe Bengt Per
Marcin Mikael Fredrik Jason
Marcin Mikael Fredrik Jason

Who are you?

Who are you?

Who are you?

Who are you?

Who are you?

Who are you?

Who are you?

Who are you?

Who are you?

Who are you?

In what way do you expect the course will benefit you?

Participants expect the course to deepen their understanding of neural nets and deep learning techniques, provide hands-on experience, and teach them how to apply these techniques to their specific research fields. Some participants hope to learn how to recognize when and how to use these techniques to improve their data analysis, while others hope to develop their own versions of neural net tools.

(Summary of your answers, generated by ChatGPT)

Course website

https://uppsala.instructure.com/courses/75565

  • Learning outcomes
  • Prerequisties/preparations
  • Discussions
  • Schedule / Modules

So, this course is about Artificial Neural networks (Anns)

In [5]:
import sys, os
sys.path.append(os.path.abspath(os.path.join('..', 'common_assets')))
import numpy as np
import matplotlib.pyplot as plt
from draw_neural_net import draw_neural_net

layer_sizes = [ 4, 3, 2, 3, 4 ]
weights = [
    np.array(
        [
            [ "w_{1,1}", "w_{1,2}", "w_{1,3}"],
            [ "w_{2,1}", "w_{2,2}", "w_{2,3}"],
            [ "w_{3,1}", "w_{3,2}", "w_{3,3}"],
            [ "w_{4,1}", "w_{4,2}", "w_{4,3}"]
        ]
    ),

    np.array(
        [
            [ "w_{1,1}", "w_{1,2}"],
            [ "w_{2,1}", "w_{2,2}"],
            [ "w_{3,1}", "w_{3,2}"]

        ]
    ),

 #   np.array(
 #       [
 #           [ "w_{1,1}", "w_{1,2}"],
 #           [ "w_{2,1}", "w_{2,2}"],
 #       ]
 #  ),
        np.array(
        [
            [ "w_{1,1}", "w_{1,2}", "w_{1,3}"],
            [ "w_{2,1}", "w_{2,2}", "w_{2,3}"]
        ]
    ),
        np.array(
        [
            [ "w_{1,1}", "w_{1,2}", "w_{1,3}","w_{1,4}"],
            [ "w_{2,1}", "w_{2,2}", "w_{2,3}","w_{2,4}"],
            [ "w_{3,1}", "w_{3,2}", "w_{3,3}","w_{3,4}"]
        ]
    )
]
biases = [
    np.array(
        ["b_1", "b_2","b_3"]
    ), 
    np.array(
        ["b_1", "b_2"]
    ), 
    np.array(
        ["b_1", "b_2", "b_3"]
    ), 
    np.array(
        ["b_1", "b_2", "b_3", "b_4"]
    ), 


]

figAE = plt.figure(figsize=(8,5))

ax = figAE.gca()
ax.axis('off')

draw_neural_net(ax, layerSizes=layer_sizes) #, weights= weights, biases=biases)

ANN2

ANN2

.... What is that?

AI vs ML vs ANN vs DL

ai_ml_ann_dl

(taken from https://medium.com/ai-in-plain-english/artificial-intelligence-vs-machine-learning-vs-deep-learning-whats-the-difference-dccce18efe7f)


AI -- Artificial Intelligence

  • The science and engineering of creating intelligent machines

ML -- Machine learning

  • Subset of AI
  • Computer algorithms that learn/improve from experience

ANN -- Artificial Neural Networks

  • Subset of ML
  • Model of human neural networks
  • Networks built up by artificial neurons

DL -- Deep Learning

  • Subset of ANN (? -- depend on definition of ANN)
  • ANN with more than one hidden layer

Where does ANN come from?

History

First wave of ANN (funding)

  • 1943 First model of biological neruon (McCulloch & Pitts)
  • 1949 Hebbian Learning rules for ANN, (Donald Hebb)
  • 1951 First ANN (Minsky)
  • 1958 The Perceptron -- linear classification
  • 1969 Limitations of the perceptron (Minsky & Papert)







Neuron

(taken from https://www.sciencedirect.com/topics/neuroscience/perceptron)

Neuron

(taken from https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon)

History

Second wave of ANN (funding)

  • (1975 Back propagation algorithm (Werbos))
  • 1986 Back propagation rediscovered and sigmoid activation -- ANNs in vogue again
  • 1985-87 ANN conferences, societies started (IEEE, INNS)
  • Vanishing gradient problem
  • SVM outcompetes ANNs, ANNs survives in protein structure prediction (e.g., CASP)

Neuron

(taken from https://www.kdnuggets.com/2019/01/backpropagation-algorithm-demystified.html)

Neuron

(taken from https://predictioncenter.org)

History

Third wave of ANN (funding)

  • GPUs and parallelization
  • "Deep Learning" buzzword and performing well in contests
  • 1997 Recursive Neural Nets. LSTMs (Smidhuber & Hochreiter)
  • 2000 ReLU (Hahnloser et al.?)
  • 2011 CNNs
  • 2020 AlphaFold2 souvereign at CASP14
  • ANNs are everywhere!

Neuron

Neuron

(taken from https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53)

Applications -- general

Written and spoke language recognition mnist

(taken from Mouret, Jean-Baptiste & Doncieux, Stéphane. (2008). Evolutionary Intelligence

Risk predictions mnist

(taken from https://spectrum.ieee.org/computing/embedded-systems/bringing-big-neural-networks-to-selfdriving-cars-smartphones-and-drones)

Applications -- general

Forecasting, weather, business

Neuron

(taken from https://www.gjesm.net/article_23079.html)

Face recognition and generation

mnist

(taken from https://medium.com/syncedreview/gan-2-0-nvidias-hyperrealistic-face-generator-e3439d33ebaf)

Applications -- Biosciences

Protein structure prediction Neuron

(taken from https://www.theverge.com/2020/12/1/21754310/deepmind-alphafold-ai-protein-folding-casp-competition)

Bioimaging diagnostics bioimaging

(taken from https://peerj.com/articles/6201/)

Applications -- Biosciences

Spatial transcriptomics Neuron (taken from https://www.uu.se/en/news-media/press-releases/press-release/?id=5226&typ=pm⟨=en)

End of introduction